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The  generalized  least-squares  problem,  in  which  an  observation  vector  satisfies  a  set  of  equations 
that  mav  be  nonlinear  and  implicit,  and  all  components  may  be  subject  to  errors,  can  be  solved  as  a 
constrained  minimization  problem  When  the  analysis  is  specialized  to  the  important  case  of  one¬ 
dimensional  curv  e  fitting  to  measurements  where  both  variables  contain  errors,  it  becomes  similar 
to  the  effective  variance  method.  A  standard  least -squares  computer  program  can  be  used  to  apply 
the  new  method;  the  results  are  superior  to  these  of  the  effective  variance  technique  A  simple 
geometrical  construction  illustrates  the  principles  of  the  new  method. 


1.  INTRODUCTION 

A  recent  paper  by  Jefferys1  considered  the  generalized 
least-squares  problem  A  v  ector  of  observations  satisfies  a 
set  of  equations  that  may  be  nonlinear  and  is  not  necessar¬ 
ily  solvable  in  the  explicit  form  e  =  /l.x  i;  all  of  the  compo¬ 
nents  of  the  v  ector  of  observations  may  be  subject  to  mea¬ 
surement  errors.  The  standard  least-squares  regression 
technique,  even  when  extended  to  nonlinear  models,  con¬ 
tains  the  implicit  assumption  that  a  is  free  of  error,  this  is 
hardly  ever  true  in  practice. 

The  method  that  Jefferys  presents  is  more  general  than 
the  conventional  method  and  avoids  the  latter's  inadequa¬ 
cies  in  formulation.  Also.  Jefferys  makes  two  very  impor¬ 
tant  points; 

1 1 1  The  standard  method's  assumption  that  one  variable 
is  subject  to  error  and  the  other  is  error-free,  when  that 
assumption  is  inappropriate,  produces  biased  estimates  of 
the  fit  parameters.  (Generalizations  to  higher  dimensiona¬ 
lity  in  all  variables  are  included.  (The  bias  is  given  explicitly 
bv  Jefferys  for  the  case  of  a  straight  line  through  the  origin 
where  a.  and  not  y.  contains  all  the  measurement  uncer¬ 
tainty 

i2(  The  major  difference  between  the  standard  least- 
squares  solution  procedure  and  the  generalized  method  is 
that  in  the  latter  the  expressions  used  in  solving  for  the 
result  must  be  recomputed  each  iteration  using  the  best 
n.e.,  revised)  estimate  of  the  observation  vector  as  well  as 
improved  estimates  of  the  fit  parameters.  This  is  a  natural 
outgrow  th  of  the  notion  that  the  least-squares  process  is  a 
method  of  improving  the  observations  via  the  mathemat¬ 
ical  model  :  How  ever,  the  point  may  not  be  obv  ious  if  one 
takes  the  common  view  of  least  squares  solely  as  a  method 
of  parameter  determination. 

Other  authors  have  treated  the  generalized  least-squares 
problem;  some  have  published  iterative  algorithms  for  fit¬ 
ting  arbitrary  functions  to  observation  vectors  all  of  whose 
components  may  possess  uncertainty  las  does  Jefferys). 1  '' 
The  principles  have  been  discussed  hefore  in  the  American 
Journal  of  Physics  ' '  Yet  those  principles  and  techniques 
seem  not  to  be  well-known.  In  particular,  the  results  ob¬ 
tained  by  Jefferys1  are  remarkably  similar  to  those  of  Britt 
and  Lueeke  4  However,  Jefferys  appeared  to  be  unaware  of 
the  existence  of  the  Britt-Luecke  paper,  or  indeed  of  any  of 
the  other  works  referred  to  by  the  present  paper  except  Ian 
earlier  edition  of  i  the  book  by  Deming.  (There  is  no  inten¬ 
tion  to  imply  anything  more  than  that  the  respective  auth¬ 
ors  approached  the  problem  in  the  same  way.  In  f..ct,  u  is 
•ig-.ifi.ant  'ha'  Jefferys  was  apparently  not  aware  of  the 


other  work,  because  that  supports  the  contention  that  such 
methods  are  not  well  known. I 

The  work  by  Jefferys  is  highlighted  here  because  the  ap¬ 
proach  is  general,  and  the  formulation  is  concise  and  com¬ 
plete.  In  the  next  section  the  results  presented  by  Jefferys 
will  be  summarized,  using  the  same  notation.  Following 
that,  it  w  ill  be  shown  how  the  formalism,  when  applied  to 
an  important  special  case,  results  in  an  algorithm  that  has 
several  advantages.  That  algorithm  has  pedagogical  and 
practical  value  because  it  is  particularly  easy  to  use,  gives 
better  results  than  "effective-variance"  techniques,*  and 
has  a  convenient  geometrical  interpretation. 

II.  FORMULATION 
A.  General 

Jefferys1  formulated  the  least-squares  problem  as  the 
minimization  of  a  particular  quadratic  form. 

v,  -  Iv'ct  'v.  111 

subject  to  the  constraints 

fix  -  v.ai  =  0.  (2 1 

giai  --  0.  (3) 

In  these  equations  a  is  a  vector  of  observations  with  covar¬ 
iance  maiiix  las.xuined  known  at  least  up  to  a  constant  fac¬ 
tor^.  The  superscript  "t "  in  Lq.  Ill  indicates  transpose. 
Equation  1 2)  is  a  set  of  equations  of  condition,  in  which  f  is  a 
vector  function  of  its  arguments,  v  is  the  vector  of  actual 
residuals  li.e.,x  =  x  -+■  v  would  have  been  the  actual  obser¬ 
vations  if  there  were  no  errorsl,  and  a  is  a  vector  of  param¬ 
eters.  The  same  v  appears  in  Eq.  1 1 1.  Equation  |3|  is  a  set  of 
constraints  on  the  parameters  a.  In  all  equations,  a  caret  I  A  | 
above  a  symbol  denotes  an  estimate  of  the  quantity  repre¬ 
sented  by  that  symbol 

A  constrained  minimization  problem  can  be  solved  con¬ 
veniently  by  the  method  of  Lagrange  multipliers,  and  that 
is  the  approach  Jefferys  chose.  The  statement  of  the  prob¬ 
lem  becomes:  Find  estimates  v  and  a  of  v  and  a,  and  k  anJ  ,2 
of  the  Lagrange  multiplier  vectors  k  and  ji.  which  mini¬ 


mize 

.S'  =  !v'(t  'v  +  f  lx.aifi  +  g'laiX,  |4) 

where  x  +  v  has  been  abbreviated  by  x.  The  minimization 
produces  the  normal  equations 

a  'v  +  fx ix.al|i  =  0.  1 5a) 

fjx.ai|l  -v  g^(a|A.  0,  |5b| 

flx.al  -  0  (5c) 
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In  these  equations  the  subscripts  x, a  denote  partial  deriva¬ 
tives  w  ith  respect  to  the  indicated  variables.  The  symbol  f( . 
for  instance,  is  v  ery  compact  notation  for  the  matrix  whose 
i.j  element  is  df  /  3x  ,  evaluated  at  x  --  x. 

The  solution  to  Eqs.  i5ai-l5dl  will  give  the  solution  to  the 
posed  problem,  if  it  exists.  However,  these  equations  are  in 
general  nonlinear  The  next  step  in  Jeffery  s'  approach  is  to 
apply  Newton's  method  to  linearize  the  equations  about  an 
approximate  solution.  Let  the  approximate  solution  be  de¬ 
noted  by  lx.  ai.  and  let  the  linear  estimate  of  the  corrections 
be  denoted  by  le.8i.  iThe  reader  should  note  in  particular 
that  the  solution  includes  all  components  of  x.  as  well  as  (he 
parameters  a.  and  the  distictton  between  €  and  v  should  be 
clear,  i  Then  Eqs.  i5ai-i5d)  become 


cr  'lv  -  ci  *  /’  p.  =  0. 

l6ai 

/» P  *  A  =  o. 

1 6b  | 

f  ~  fxe  -  f„8  -  0. 

1 6c 1 

g  -  ga  8  ^  o 

Itidl 

At  this  point,  in  anticipation  of  the  special  case  to  be  dis¬ 
cussed  in  the  next  section,  the  set  of  constraints  on  the 
parameters  [Eq.  i3i]  will  be  dropped.  This  causes  a  slight 
simplification  in  the  equations  just  stated:  the  second  term 
in  Eq.  ibb.i  and  all  of  Eq.  i6di  vanish,  and  the  Lagrange 


multipliers  k  are  no  longer  needed. 

Continuing  the  solution,  if  Eq.  1 6a I  is  solved  for  e.  the 
result  is 

t  =  -  v  trfj  |i.  (7) 

When  this,  in  turn,  is  substituted  into  Eq.  Ibci. 

"f-  f,v  -  fxofx|i  -  f„8  =  0,  |8l 

which  can  be  solved  for  the  Lagrange  multipliers 

p  =  Wlf  -  f,  v  -+-  f„8l.  (9) 

where  the  weighting  matrix  \\  is 

W-If.af'l  (10) 

The  (simplified!  Eq.  1 6b i  may  now  be  solved  to  obtain  the 
reduced  normal  equation 

f.Wf.8=  -  f'\V<t>,  (111 

where 

6  =  f  -  f,  v.  I12i 

The  solution  to  Eq.  1 1 1 1  can  be  used  to  “update"  the  param¬ 
eter  estimates, 

a„,„  =  a  *  6.  113) 

But  in  order  to  complete  the  iterative  process,  it  is  also 

necessary  to  update  x, 

x,u.,  =  x  -  k  (14) 

Equation  (6a)  may  be  used  to  find  k  After  substituting  |i 
from  Eq.  |9|  and  performing  some  manipulations,  the  final 
result  is 

v„...  •  115) 

where 

vnr„  =  -  of, W'tS  *  (lb; 

This  is  equivalent  to  Eq.  (13). 

Jefferys  suggests  using  x  =  x  as  an  initial  estimate  of  x 


lagain,  it  must  be  emphasized  that  x  will  change),  along 
with  some  vector  a  of  initial  parameter  "guesses”  for  a.  The 
solution  process  consists  of  solv  ing  Eq.  (Ill  for  8.  using  the 
current  estimates  of  everything,  then  substituting  the  re¬ 
sulting  8  into  Eq.  1 161  to  obtain  vm.„  .  Equations  ( 1 3 1  and 
1 1  ?)  then  give  improved  values  of  a  and  x.  This  constitutes 
one  iteration  Successive  iterations  are  performed,  if  neces¬ 
sary.  until  a  set  of  convergence  criteria  are  met.  or  until  a 
prespecified  large  number  of  iterations  are  performed  with¬ 
out  reaching  convergence.  The  latter  method  of  stopping 
the  calculation  may  be  necessary  for  one  of  two  reasons:  If 
the  problem  is  sufficiently  complex  and  the  initial  estimates 
are  too  far  away,  th’s  linearized  algorithm  may  not  con¬ 
verge  to  the  solution  at  all.  Or  even  if  it  does  converge,  the 
finite  t  limited  I  precision  of  the  machine  may  not  permit 
satisfaction  of  a  sev  ere  convergence  test. 

The  reason  for  duplicating  Jefferys'  formulation  here  in 
such  detail  is  so  that  we  can  go  from  the  general  to  the 
specific  in  the  next  section  w  ith  a  minimum  of  effort  but 
w  ith  confidence  in  the  v  alidity  of  the  results.  The  resulting 
algorithm  w  ill  be  amenable  to  a  particularly  simple  imple¬ 
mentation. 


B.  One-dimensional  curve  fitting 

The  preceding  section  has  been  entirely  general.  It  treat¬ 
ed  the  least-squares  fitting  of  any  relationship  of  the  form 
fix)  =  0.  where  x  is  a  vector  of  observations  land  the  depen¬ 
dence  on  the  adjustable  parameters  has  been  suppressed). 
By  far  the  most  frequent  application  of  least  squares  is  to 
cases  in  which  the  functional  relationship  has  the  explicit 
form 


)’  =  uix  |. 


(17) 


where  both  x  and  y  are  scalars.  There  is  some  slight  confu¬ 
sion  in  notation:  the  components  of  x  for  the  :th  observa¬ 
tion  are  U,,y,  |(f  =  1,2 . A’,  where  A*,  is  the  number  of  ob¬ 

servations).  The  vector  function  fix)  has  .V  components, 
f,  =  ulx,  ,a|  —  y, .  The  vector  of  parameters  a  has  m  compo¬ 
nents.  We  assume  uncorrelated  measurements  lx,  ,y, ),  so 
the  matrix  <r  is  diagonal. 

A  few  of  the  matrices  will  be  displayed  explicitly,  along 
with  their  dimensions: 


o.i  ‘  0 

0  oy, : 

0  0 


0 

0 

o\,  • 

0 


fx  = 


du , 
Ax  | 

0 


1  0 

Jx, 


0 

0 

0 

<7, A 


1 2  A’  x  2A ) 


(18) 


-  1 


[N  X2A) 


(19) 
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du, 

du , 

du 

da, 

da , 

3a  n 

du. 

du , 

du 

da | 

da: 

dan 

du  s 

du  v 

du , 

da | 

da : 

da„ 

The  partial  derivatives  in  Eqs.  1 1 9|  and  '20l  are  evaluated  at 
lx.ai  at  each  stage  of  the  iterative  calculation.  The  vector  of 
actual  residuals  is 


-  *1 


f  2.V  • 


i2 1 1 


L  Tv  -  v  J 

In  Eq.  ( 2 1 I,  v\  =  mu,  i. 

Using  these  relations,  it  can  be  seen  that  f,  <jf'x  is  now  an 
.V  •  .V  diagonal  matrix  whose  /th  diagonal  element  is 
l du  / dx.\'crx.  cru.  Consequently,  Eq.  (IOi  shows  that 
the  weighting  matrix  \V  is  also  an  A  ■  .V  diagonal  matrix, 
whose  /th  diagonal  element  is 


W  = 


(221 


And  the  vector  <J>,  defined  in  Eq.  ll2l,  is  the  X  x  1  vector 
whose  /th  element  is 


6,  =  -  \y;  -  uix;,ai  -  — — w,  -  x,  ij.  (23) 

[Once  again,  the  partial  derivatives  in  both  equations  are 
evaluated  at  (Jc.ai.J 

Reference  to  Fig.  1  (which  suppresses  the  index  /)  shows 
that  6,  is  the  negative  of  the  quantity  shown  as  d  (i.e.,  d, ): 

6  —  —  d, .  (24) 

Note  furthermore  that,  to  a  good  approximation. 

6,^  -  R,  =  u\x,  ,ai  —  y, .  (25) 


Figure  1  depicts  the  region  near  one  observation  in  a 
space  with  (x,y)  normalized  so  that  the  variances  at  that 
point  are  unity.  From  Eqs.  ( 1 1  and  (2 1 )  it  can  be  seen  that  the 
sum  of  squares  5„  reduces  to  a  sum  of  terms  of  the  form 
\D,  \  where  the  line  of  length  D,  is  a  perpendicular  from 
\x,y\  to  the  curve  at  (x,u|x)).  This  provides  a  convenient 
geometrical  interpretation  of  generalized  least  squares  for 
this  important  special  case. 

It  is  now  possible  to  carry  out  Jefferys’solution  proce¬ 
dure.  The  first  step  is  to  solve  Eq.  Ill)  for  6.  In  the  present 
case. 


mw*),  =  -  I—  W„Rn.  (26b) 

„  ,  oa!  „  |  ddj 

Consequently,  with  the  second  form  of  Eq.  (26b),  Eq.  ( 1 1 )  is 
exactly  the  equation  obtained  for  the  successive  differential 
corrections  to  the  parameters  in  the  standard  least-squares 
method,  with  two  exceptions: 

(ll  All  of  the  partial  derivatives,  with  respect  to  both  x 
and  a,  are  evaluated  at  (Jc.a)  instead  of  U,a|. 


Fig.  1  Least-squares  geomcii  v  in  transformed  space  with  unit  variances, 


|2|  The  weights  in  the  W  matrix  are  of  the  form 


i 


(27) 


These  are  the  effective  variance  weights.1'  They  have  been 
shown  previously  to  be  the  appropriate  weights  to  use  for 
least  squares  when  both  x  andy  observations  are  subject  to 
error. IJ  |S  However,  the  effective-variance  algorithm  de¬ 
scribed  in  those  references  neglected  the  other  “exception” 
above.  Consequently,  it  has  been  found  that  it  almost  never 
gives  an  exact  least-squares  solution. Ih 

Because  the  method  derived  in  this  section  is  formally 
identical  to  standard  least  squares,  with  the  exceptions  not¬ 
ed  above,  it  has  the  very  important  practical  advantage  that 
it  can  be  implemented  simply  by  making  slight  modifica¬ 
tion  to  existing  curve  fit  computer  software.  The  standard 
method  effectively  assumes  that  there  are  no  x  residuals — 
in  other  words,  that  there  is  no  uncertainty  in  thex  obser¬ 
vations.  Consequently,  it  is  not  applicable  when  there  are  x 
uncertainties.  When  it  is  applied  in  such  cases  it  produces 
biased  fits,  as  Jefferys  showed. 1  However,  with  the  relative¬ 
ly  minor  changes  indicated  above,  a  computer  program 
that  was  designed  for  the  standard  method  can  be  used  for 
the  general  problem. 

The  approximation  of  replacing  d,  by  R,  still  remains,  so 
it  may  appear  that  the  result  of  all  the  foregoing  algebra 
was  just  to  replace  one  approximate  method — effective 
variance — with  another.  But  it  will  be  seen  that  the  new 
method  is  a  much  better  approximation  than  the  effective 
variance  technique,  while  retaining  the  letter’s  ease  of  im¬ 
plementation.  Furthei,  Fig.  1  provides  a  consistent  geo¬ 
metrical  interpretation  of  the  method  that  is  of  pedagogical 
value.  The  use  of  the  Figure  will  continue  below. 

The  one  missing  element  is  a  method  of  improving  the  x 
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estimates,  in  order  that  we  may  evaluate  the  partial  deriva¬ 
tives  at  x  rather  that  x.  From  Eqs.  (7)  and  ( 1 4) -( 16), 

€  =  —  v  —  orfx|i  =  x  —  x  —  <rfxW|<i>  +  f48i.  |28| 

Equation  1 28)  is  for  the  general  case.  If  we  now  specialize  to 
one-dimensional  curve  fitting  and  drop  the  fu  5  term,  which 
is  a  first-order  correction  to  the  "updated"  a  li.e..  a  -f-  5). 
then  the  correction  to.v  is 


(S.v  =  .v 


.  du 

cr  — -W  i 
dx.  '  ' 


—  .V 

(TXi 

du  ( 

du 

— i  “ 

-  y.  -  — H.t 

dx  \  ' 

dx, 

\(du\ 

- 

KsrJ 

ax.  ~  avl  2 
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where  u.  stands  for  u\x,  .a).  If  w  e  now  define  the  separate  x 
and  v  weights 

H\  =  1/ctx;:.  =  I /a,,,’.  (30) 

then  Eq.  (29)  becomes 


dx. 


—  u , 

du 

Ih 


du , 

i- — 

dx 


Wu  \x, 


W. 


(31) 


Figure  1  provides  an  immediate  geometrical  interpreta¬ 
tion.  Assume  that  a,  =  a,  =  1.  I  Again,  we  suppress  the 
index  i.i  Then  the  line  of  length  D  is  perpendicular  to  the 
curve  and  has  the  equation 


.VI  .XI  --  V  - 


I 

i.t  -  .vi. 


1 32 1 


Now  suppose  that  we  have  an  x„  that  is  an  approximation 
and  wish  to  find  a  better  one,  x.  The  curve  near  Jc„  is  given 
by.  to  first  order, 

.  .  .  du  x  „  , , 

utx.ai  =■  u  i.Vn.ai  —  — dx,  i33; 

dx 


where  dx  =  x  -  x„.  The  needed  correction  from  ,v(l  to  ,v  is 
given  by  the  simultaneous  solution  of  Eqs.  |32)  and  1 33 1. 
The  result  is 


When  the  variables  are  "unnormalized"  (transformed  to 
arbitrary  variances)  and  allowance  is  made  for  the  slight 
changes  in  notation,  Eq.  (34)  becomes  identical  to  Eq.  (31). 


Following  Jefferyx'  suggested  algorithm  of  the  preceding 
section,  a  sketch  of  the  solution  procedure  for  one-dimen¬ 
sional  curve  lilting  when  x  and  v  contain  measurement  er¬ 
ror  is  as  follows:  Eq.  1 3 1 )  gives  the  updates  to  the  indepen- 


Tablc  1  Pearson's  data  and  York's  weights 
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dent  variable.  With  the  weights  (22).  Eqs.  (26a l  and  i26bi 
give  the  factors  which  go  into  Eq.  (Ill  for  the  parameter 
updates.  In  this  case  Eq.  (Hi  is  exactly  that  used  to  give 
parameter  updates  in  standard  least  squares,  except  that 
the  partial  derivatives  are  evaluated  at  the  improved  x,  and 
the  special  choice  of  weights  (22)  is  used.  Equation  1 1 1 )  is 
iterated  until  a  convergence  test  is  passed  or  an  iteration 
limit  is  reached.  The  updates  will  change  the  weights,  so 
the  entire  procedure  (variable  updates  and  parameter  up¬ 
dates]  is  iterated  until  a  suitable  convergence  criterion, 
such  as  the  effective  variance  criterion  proposed  by  Clut- 
ton-Broek.M  is  satisified.  It  may  also  be  desirable  to  check 
the  convergence  of  the  independent  variable  estimates.  The 
overall  sum  of  squares  of  residuals  is  another  candidate  for 
a  convergence  check,  since  that  is  what  is  supposed  to  be 
minimized. 

III.  EXAMPLES  AND  COMPARISON  OF 
METHODS 

The  method  described  in  the  last  section  was  pro¬ 
grammed  and  applied  to  sev  eral  cases.  The  same  program 
("built"  around  a  standard  least-squares  program  I  was 
used  to  perform  least  squares  and  effective  variance  fits. 
The  results  are  summarized  below,  along  with  published' 
exact  generalized  least-squares  results. 

The  cases  consisted  of  polynomial  fits  to  data  given  by 
Pearson,  who  analyzed  the  problem  of  performing  linear 
fits  to  measurements  with  error  in  all  variables  as  long  ago 
as  1901. 17  The  weights  [cf.  Eq.  (30)]  used  consisted  of  two 
sets:  unit  weights  on  both  x  and  y,  all  points,  and  a  set  of 
weights  used  with  Pearson’s  data  by  York.1*  Pearson's  data 
and  York's  weights  are  listed  in  Table  I. 

Table  II  shows  the  results  of  fits  of  a  straight  line, 
v  =  6i,  4-  a,x.  to  the  data  of  Table  I.  X  was  calculated  from 

S=  ]T  I  --NF  +  Wjy,  —  y,  l:  ] .  (351 


Table  li  Linear  tils  to  Pearson’s  data  with  York’s  weights 


Calculated 

value 

Standard 
least  squares 

Kffective 

variance 

New 

algorithm 

L.xact 

solution 

a 

h  HX)  10445 

5  .14605212 

5.47WI025 

5  47441022 

a 

0  610812467 

0.461448885 

0  480511415 

0480511407 

V 

16  285266046 

1  |  45644408 

1 1  8661511441 

1 1.8661511441 
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Table  III  Polynomial  tils  to  Pearson's  data  with  unit  weights  or  York's  weights 


Polynomial 

degree 

Weight-. 

Standard 
least  squares 

Effective 

variance 

S  lor 

New 

algorithm 

Exact 

solution 

i 

Inn 

0.6201  14636654 

0 620114636554 

0.61X572754437 

0  618572754437 

i 

York 

16.2)15266046 

11. 45644  408 

I  1.866353 1441 

)  1  8663531441 

3 

tnu 

0.487,1164486X3 

0.486175535205 

0.485155481444 

0  485152486427 

3 

York 

1 1  8X32634403 

10  5414454561 

10  4878374448 

10  4864040577 

5 

Inn 

().453(XX)7 17248 

0.453(XX)307864 

0  450324466231 

0  450325667217 

which  is  equivalent  to  Eq.  1 1 1  except  for  the  factor  of  1/2. 
The  v  alues  x  are  calculated  directly  as  a  by-product  of  the 
calculation  for  the  fits  by  the  new  algorithm  and  the  exact 
so'i'tion  For  the  other  fits,  Eq.  1 3 1 1  was  used  to  find  thex, 
consistent  with  the  fit  parameters,  in  order  to  be  able  to 
calculate  5.  The  results  show  that  the  standard  least- 
squares  solution  is  seriously  in  error  and  the  effective  vari¬ 
ance  solution  is  substantially  better,  but  the  result  of  the 
new  algorithm  is  tdentical  to  the  exact  solution  within  the 
precision  of  the  calculation.  The  latter  happens  because  no 
approximation  is  made  by  the  new  algorithm  for  linear  fits, 
since  d.  is  exactly  the  same  as  R..  The  effective  variance 
algorithm  is  not  exact;  since  du./da:  =  x,  the  failure  to 
update  x  in  the  effective  variance  calculation  prevents  the 
normal  equation  from  producing  a  true  minimum  even 
though  the  weights  are  formally  correct. 

Table  Ill  shows  the  values  of  S  obtained  using  each  of  the 
fit  procedures  for  various  polynomial  fits  to  Pearson's  data 
with  either  unit  weights  or  York's  weights.  An  interesting 
situation  occurs  in  the  case  of  straight-line  fits  with  unit 
weights.  The  value  of  S  for  the  effective  variance  fit  is  the 
same  as  that  for  the  standard  least-squares  fit.  The  two  fits 
are  actually  the  same:  the  sets  of  coefficients  are  identical, 
also.  This  result  is  not  unexpected.  Chandler1''  showed 
that,  for  straight-line  fits  with  =  a  and  a„  =  r  for  all  /. 
the  effective  variance  algorithm  gives  the  same  result  as 
standard  least  squares.  That  solution  is  seen  to  be  less  than 
optimum,  but  the  result  of  the  new  algorithm  is  again  iden¬ 
tical  to  the  exact  solution. 

For  polyomial  fits  of  higher  degree,  the  new  algorithm  is 
not  exact.  It  is  very  nearly  so.  however,  and  it  outperforms 
the  effective  variance  algorithm  in  every  case.  On  a  linear 
scale  w  hich  measures  the  percentage  decrease  inS  from  the 
standard  least-squares  result  to  the  exact  solution,  the  ef¬ 
fective  variance  method  attained  an  average  "score"  of 
4X.63K'T ,  while  the  new  algorithm  averaged  99.9279f. 

The  relative  differences  between  coefficient  values  for 
the  different  methods  were  even  greater  than  the  corre¬ 
sponding  differences  between  .V  values.  In  particular,  in  the 
quintic  polynomial  fits  the  standard  least  squares  and  effec¬ 
tive  v  ariance  algorithms  yielded  some  coefficients  with  sign 
differences  compared  to  the  exact  solution,  while  the  new 
algorithm's  result  agreed  very  closely.  The  new  algorithm 
has  also  been  used  successfully  to  fit  functions  that  depend 
nonhnearly  on  the  fit  parameters.  However,  the  cases  pre¬ 
sented  are  widely  used  in  the  literature  as  standard  test 
cases,  permitting  easy  comparison  with  other  methods. 

IV.  CONCLUSION 

The  standard  least-squares  procedure  should  not  be  used 
to  perform  curve  fits  when  both  x  and  v  contain  measure¬ 


ment  uncertainty,  because  the  underlying  assumptions  are 
violated  and  the  method  produces  biased  estimates  of  the 
fit  parameters.'  The  effective  variance  technique,  which 
has  been  recommended  in  this  Journal  for  use  in  such 
cases,"  ' s  is  usually  better  but  is  still  not  exact.  A  new  meth¬ 
od.  which  can  be  obtained  as  the  result  of  specializing  an 
analysis  of  the  generalized  least-squares  problem'  to  the 
case  of  one-dimensional  curve  fits  to  uncorrelated  observa¬ 
tions.  gives  better  results  but  is  no  harder  to  use.  One  slight 
approximation  makes  the  new  method  formally  identical 
to  the  effective  variance  approach  with  one  enhancement, 
so  it  shares  the  latter  method’s  advantage  that  it  can  be 
applied  using  a  standard  least  squares  computer  program. 
Tests  prove  that  the  new  method  is  exact  for  linear  regres¬ 
sion  iwhile  the  effective  variance  method  is  no  better  than 
standard  least  squares  when  all  the  x  uncertainties  are 
equal  and  all  they*  uncertainties  are  equal)  and  performs 
better  than  the  effective  variance  method  in  other  cases. 
There  is  a  convenient  geometrical  interpretation;  in  a  sense 
the  new  method  is  completely  described  by  Fig.  1. 

The  new  method  is  genuinely  easy  to  apply  using  a  stan¬ 
dard  least-squares  algorithm.  A  microcomputer  imple¬ 
mentation  can  be  used  to  fit  arbitrary  functions.  The  author 
has  programmed  the  method  for  straight  lines  on  the  Texas 
Instruments  TI-59  programmable  pocket  calculator,  using 
the  calculator's  built-in  straight-line-fit  capability.  That 
program  is  available  on  request. 

The  method  introduced  in  the  paper  is  an  outgrowth  of 
procedures  developed  for  calibration  of  the  measurements 
made  in  Skylab  Experiment  S-233.  photographic  photome¬ 
try  of  Comet  Kohoutek  observations. 
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